Learning the Reward Model of Dialogue POMDPs from Data

Spoken language communication between human and machines has become a challenge in research and technology. In particular, enabling the health care robots with spoken language interface is of great attention. Recently due to uncertainty characterizing dialogues, there has been interest for modelling the dialogue manager of spoken dialogue systems using Partially Observable Markov Decision Processes (POMDPs). In this context, we would like to learn the reward model of dialogue POMDPs from expert’s data. In a previous paper, we used an unsupervised learning method for learning the states, as well as the transition and observation functions of the dialogue POMDPs based on human-human dialogues. Continuing our objective of learning the components of dialogue POMDPs from data, here we introduce a novel inverse reinforcement learning (IRL) algorithm for learning the reward function of the dialogue POMDP model. Based on the introduced method, and from an available corpus of data, we construct a dialogue POMDP. Then, the learned dialogue policies, based on the learned POMDP, are evaluated. Our empirical evaluation shows that the performance of the learned POMDP is higher than expert performance in non, low, and medium noise levels, but the high noise level. At the end, current limitations and future directions are addressed.